Control of autonomous vehicle based on fusion of pose information and visual data

ABSTRACT

Embodiments of the present application disclose a positioning method and apparatus, an autonomous driving vehicle, an electronic device and a storage medium, relating to the field of autonomous driving technologies, comprising: collecting first pose information measured by an inertial measurement unit within a preset time period, and collecting second pose information measured by a wheel tachometer within the time period; generating positioning information according to the first pose information, the second pose information and the adjacent frame images; controlling driving of the autonomous driving vehicle according to the positioning information. The positioning information is estimated by combining the first pose information and the second pose information corresponding to the inertial measurement unit and the wheel tachometer respectively. Compared with the camera, the inertial measurement unit and the wheel tachometer are not prone to be interfered by the external environment.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese Patent Application No.202010497244.0, filed on Jun. 2, 2020, which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

Embodiments of the present application relate to the field of computertechnologies, especially to the field of autonomous driving technology,in particular to a positioning method and apparatus, an autonomousdriving vehicle, an electronic device and a storage medium.

BACKGROUND

With the development of the autonomous driving technology, how torealize accurate positioning of vehicles to ensure the safety ofvehicles and pedestrians has become an urgent problem to be solved.

In the prior art, positioning methods for autonomous driving vehiclesare mainly based on a visual odometer, while the existing visualodometers mainly accomplish positioning by monocular cameras orbinocular cameras, such as estimating position and posture byaccumulating motion between frames.

However, in the implementation of the proposal, the inventor found atleast the following problem: the accuracy of the estimated position andposture is low since the camera is prone to vibrate during motion.

SUMMARY

A positioning method and device for accurately positioning, anautonomous driving vehicle, an electronic device and a storage mediumare provided.

According to a first aspect, a positioning method is provided, which isapplied to an autonomous driving vehicle, and the method includes:

collecting first pose information measured by an inertial measurementunit within a preset time period, and collecting second pose informationmeasured by a wheel tachometer within the time period, where the timeperiod is a sampling time interval when a camera collects adjacent frameimages;

generating positioning information according to the first poseinformation, the second pose information and the adjacent frame images;and

controlling driving of the autonomous driving vehicle according to thepositioning information.

In the embodiment of the present application, the positioninginformation is estimated by combining the first pose information and thesecond pose information corresponding to the inertial measurement unitand the wheel tachometer respectively. Compared with the camera, theinertial measurement unit and the wheel tachometer are not prone to beinterfered by the external environment. Therefore, the interference ofthe positioning information by the external environment can be avoided,thereby improving the accuracy and reliability of the positioninginformation, so that the autonomous driving vehicle can drive safely andreliably.

According to a second aspect, a positioning apparatus is provided,including:

a collecting module, configured to: collect first pose informationmeasured by an inertial measurement unit within a preset time period,and collect second pose information measured by a wheel tachometerwithin the time period, where the time period is a sampling timeinterval when a camera collects adjacent frame images;

a generating module, configured to generate positioning informationaccording to the first pose information, the second pose information andthe adjacent frame images; and

a controlling module, configured to control driving of the autonomousdriving vehicle according to the positioning information.

According to a third aspect, an electronic device is provided,including:

at least one processor; and

a memory connected with the at least one processor in communication,where,

the memory stores instructions executable by the at least one processor,where the instructions are executed by the at least one processor tocause the at least one processor to perform the method according to anyone embodiment as mentioned above.

According to a fourth aspect, an autonomous driving vehicle is provided,and the vehicle includes a positioning apparatus described in the aboveembodiment, or an electronic device described in the above embodiment.

According to a fifth aspect, a non-transitory computer-readable storagemedium storing computer instructions is provided, where the computerinstructions are configured to cause a computer to perform the methodaccording to any one embodiment as mentioned above.

According to a sixth aspect, a positioning method is provided, themethod including:

collecting respective pose information measured by at least two sensorswithin a preset time, where the time period is a sampling time intervalwhen a camera collects adjacent frame images;

generating positioning information according to the respective poseinformation and the adjacent frame images.

According to technical solutions of embodiments of the presentapplication, the disadvantages of the positioning information prone tobe interfered by the external environment when the positioning iscarried out via the camera in the related technology is solved, and theaccurate positioning is realized, thus realizing the technical effect ofsafe and reliable driving of the autonomous driving vehicle.

It should be understood that the content described in this portion isnot intended to identify key or important features of embodiments of thepresent disclosure, nor to limit the scope of the present disclosure.Other features of the present disclosure will be easily understood bythe following description.

BRIEF DESCRIPTION OF DRAWINGS

Drawings are used for better understanding of the present solution anddo not limit the present application, in which:

FIG. 1 is a schematic diagram of an application scene of an embodimentof the present application;

FIG. 2 is a schematic diagram of an application scene of anotherembodiment of the present application;

FIG. 3 is a flowchart diagram of a positioning method of an embodimentof the present application;

FIG. 4 is a flowchart diagram of a positioning method of anotherembodiment of the present application;

FIG. 5 is a flowchart diagram of a method for generating fused poseinformation of the present application;

FIG. 6 is a flowchart diagram of a method for generating the positioninginformation according to adjacent frame images and the fused poseinformation of the present application;

FIG. 7 is a flowchart diagram of a positioning method of yet anotherembodiment of the present application;

FIG. 8 is a schematic diagram of the positioning apparatus according toan embodiment of the present application;

FIG. 9 is a block diagram of an electronic device according to anembodiment of the present application; and

FIG. 10 is a flowchart diagram of a positioning method of yet anotherembodiment of the present application.

DESCRIPTION OF EMBODIMENTS

The following illustrates exemplary examples of embodiments of thepresent application in combination with the drawings, in which variousdetails of the embodiments of the present application is included tofacilitate understanding thereof, and they shall be regarded as merelyexemplary. Therefore, those skilled in the art should realize thatvarious changes and modifications can be made to the embodimentsdescribed herein without departing from the scope and spirit of theembodiments of the present application. Similarly, for the sake ofclarity and conciseness, the description of well-known functions andstructures is omitted in the following.

Referring to FIG. 1 , FIG. 1 is an application scene diagram of apositioning method of an embodiment of the present application.

In the application scene shown in FIG. 1 , an autonomous driving vehicle100 runs on a straight road, and the autonomous driving vehicle 100includes a processor (not shown in FIG. 1 ), which can execute thepositioning method of embodiments of the present application todetermine corresponding positioning information of the autonomousdriving vehicle 100, and to control the autonomous driving vehicle 100to adjust driving strategy according to the positioning information.

The driving strategy is used to represent a driving state of theautonomous driving vehicle, and the driving strategy includesdeceleration, parking, acceleration, turning and the like.

For example, when the autonomous driving vehicle encounters a red light,the processor can control the autonomous driving vehicle to adjust astraight driving strategy to a deceleration driving strategy, or toadjust the straight driving strategy to a parking driving strategy; whena vehicle in front of the autonomous driving vehicle accelerates and theautonomous driving vehicle is far away from the vehicle in front, theprocessor can control the autonomous driving vehicle to adjust thestraight driving strategy to an acceleration driving strategy, and thelike.

In the application scene shown in FIG. 1 , when the positioninginformation for the autonomous driving vehicle 100 is determined by theprocessor, it will know that the autonomous driving vehicle is close toan intersection where a left turning can be performed. If a route forthe autonomous driving vehicle 100 corresponds to a left turning road,that is, the autonomous driving vehicle 100 needs to make a left turn torealize effective driving, the processor can control the autonomousdriving vehicle 100 to adjust the straight driving strategy to a leftturning driving strategy according to the positioning information.

As for the case in that the processor controls the autonomous drivingvehicle 100 to adjust the straight driving strategy to the left turningdriving strategy, please refer to the scene diagram shown in FIG. 2 fordetails.

In related technologies, when positioning an autonomous driving vehicle,a monocular camera or a binocular camera is mainly used to accomplishthe positioning, such as estimating a position and posture byaccumulating motions between frames.

However, since the camera is prone to vibrate during motion, and theacquisition of an image by the camera is prone to be affected by thesurrounding environment; as a result, by means of solutions of therelated technologies, it is easy to cause low accuracy of an estimatedposition and posture, that is, it is easy to cause the problem ofinaccurate positioning information.

In order to solve the above technical problems, the inventor of thepresent application, after paying creative work, obtains the inventionconcept of the present application: eliminating the dependence of acamera on the environment as much as possible when obtaining a picture,and correcting the positioning information effected by vibration, so asto obtain positioning information with higher accuracy.

The following illustrates in detail technical solutions of the presentapplication and how the technical solutions of the present applicationsolve the above-mentioned technical problems with respect to specificembodiments. The following specific embodiments can be combined witheach other, and the same or similar concepts or processes may not berepeated in some embodiments. Embodiments of the present applicationwill be described in combination with the drawings in the following.

Referring to FIG. 3 , FIG. 3 is a flowchart diagram of a positioningmethod of an embodiment of the present application.

As shown in FIG. 3 , the method includes:

S101: collecting first pose information measured by an inertialmeasurement unit within a preset time period, and collecting second poseinformation measured by a wheel tachometer within the time period, wherethe time period is a sampling time interval when a camera collectsadjacent frame images.

The executive body of embodiments of the present application may be apositioning apparatus, and the positioning apparatus can be a computer,a server, a processor, an on-board terminal, a remote informationprocessor (on-board T-BOX) and a chip (such as an embedded chip), etc.,which is provided on an autonomous driving vehicle.

For example, if the positioning method of an embodiment of the presentapplication is applied to the application scene shown in FIG. 1 , theexecutive body of the positioning method in the embodiment of thepresent application may be a processor.

The time period is related to the adjacent frame images collected by thecamera, which in particular can be the time period determined accordingto the corresponding time interval of the adjacent frame images. That isto say, if the time of a first frame image captured by the camera is T0,the time of a second frame image is T1, and the first frame image andthe second frame image are two adjacent frame images, and thus the timeperiod=T1−T0. The pose information includes position information andposture information.

In this step, the pose information (i.e. the first pose information)measured by the inertial measurement unit and the pose information (thesecond pose information) measured by the wheel tachometer within thetime period are collected respectively.

The inertial measurement unit (IMU) is a device for measuring a postureangle (or angular velocity), acceleration and the like of an object.Generally, one IMU includes three single axis accelerometers and threesingle axis gyroscopes, where the accelerometer detects accelerationsignal of an object on independent three-axis in a carrier coordinatesystem, and the gyroscope detects angular velocity signal of a carrierrelative to a navigation coordinate system, so as to measure the angularvelocity and the acceleration of the object in the three-dimensionalspace.

In other words, the first pose information is configured to representthe corresponding position information and/or posture information of theautonomous driving vehicle collected by the inertial measurement unit,where the posture information includes a posture angle, an angular rate,acceleration and other information. That is, in embodiments of thepresent application, the inertial measurement unit can determine theposture angle, the angular rate, the acceleration and other informationof the autonomous driving vehicle, and the relevant information measuredby the inertial measurement unit can be determined as the first poseinformation.

The wheel tachometer is also referred to as a wheel speed sensor, andthe second pose information is configured to represent the correspondingposition information and/or posture information of the autonomousdriving vehicle collected by the wheel tachometer, where the postureinformation includes speed, acceleration and other information. That is,in embodiments of the present application, the wheel tachometer isconfigured to collect the speed, the acceleration and other informationof the autonomous driving vehicle, and the collected relevantinformation can be determined as the second pose information.

It is worth noting that the terms “first” and “second” of the first poseinformation and the second pose information are used to distinguish therelevant information collected by the inertial measurement unit and thewheel tachometer.

S102: generating the positioning information according to the first poseinformation, the second pose information and the adjacent frame images.

In this step, it can be understood that the positioning information isestimated according to the first pose information, the second poseinformation and the adjacent images.

In the embodiment of the present application, the positioninginformation is estimated by combining the first pose information and thesecond pose information corresponding to the inertial measurement unitand the wheel tachometer respectively. Compared with the camera, theinertial measurement unit and the wheel tachometer are not prone to beinterfered by the external environment. Therefore, the interference ofthe positioning information by the external environment can be avoided,thereby improving the accuracy and reliability of the positioninginformation.

S103: controlling driving of the autonomous driving vehicle according tothe positioning information.

In this step, after the positioning information is determined, thedriving of the autonomous driving vehicle can be controlled, forexample, to adjust the driving strategy of the autonomous drivingvehicle. As shown in FIG. 1 , the driving strategy of the autonomousdriving vehicle is adjusted from the straight driving strategy to theleft turning driving strategy according to the positioning information,so that the autonomous driving vehicle can drive safely and reliably.

On the basis of the above analysis, the embodiment of the presentapplication provides a positioning method, which can be applied to theautonomous driving vehicle. The method includes: collecting first poseinformation measured by an inertial measurement unit within a presettime period, and collecting second pose information measured by a wheeltachometer within the time period, where the time period is a samplingtime interval when a camera collects adjacent frame images; generatingpositioning information according to the first pose information, thesecond pose information and the adjacent frame images; controllingdriving of the autonomous driving vehicle according to the positioninginformation. The positioning information is estimated by combining thefirst pose information and the second pose information corresponding tothe inertial measurement unit and the wheel tachometer respectively.Compared with the camera, the inertial measurement unit and the wheeltachometer are not prone to be interfered by the external environment.Therefore, the interference of the positioning information by theexternal environment can be avoided, thereby improving the accuracy andreliability of the positioning information, so that the autonomousdriving vehicle can drive safely and reliably.

In order to facilitate better understanding of the specific process ofgenerating the positioning information, the positioning method accordingto an embodiment of the present application is described in detail incombination with FIG. 4 . FIG. 4 is a flowchart diagram for apositioning method of another embodiment of the present application.

As shown in FIG. 4 , the method includes:

S201: collecting first pose information measured by an inertialmeasurement unit within a preset time period, and collecting second poseinformation measured by a wheel tachometer within the time period, wherethe time period is a sampling time interval when a camera collectsadjacent frame images.

Regarding the description of S201, please refer to S101, and it will notbe repeated here.

S202: fusing the first position and position and the second postureinformation to generate fused pose information.

In order to ensure the reliability of the pose information used todetermine the positioning information, in this step, the first poseinformation and the second pose information are fused.

In other words, the step may be a process of rectifying the two poseinformation mutually, so as to improve the reliability and accuracy ofthe pose information used to generate the positioning information asmuch as possible.

It will be known in combination with FIG. 5 that, in some embodiments,S202 may specifically include:

S21: obtaining a coordinate transformation parameter of the wheeltachometer relative to the inertial measurement unit.

Specifically, rotation, displacement and other parameters of the wheeltachometer relative to the inertial measurement unit can be calibrated.That is, taking the coordinate system of the inertial measurement unitas the reference and the inertial measurement unit as the origincoordinate, the rotation, the displacement and other parameters of thewheel tachometer relative to the inertial measurement unit are obtainedto determine the coordinate transformation parameter.

S22: performing a coordinate transformation on the second poseinformation according to the coordinate transformation parameter.

This step can be understood as transforming the second pose informationfrom a wheel tachometer coordinate system to an inertial measurementunit coordinate system.

Specifically, a calculation method of the transformation can refer tothe coordinate transformation formula in the prior art, which will notbe repeated here.

S23: fusing the first pose information and the second pose informationsubjected to the coordinate transformation to generate the fused poseinformation.

It is worth noting that after S22, the second pose information (i.e. thesecond pose information subjected to the coordinate transformation)taking the inertial measurement unit coordinate system as the referencecan be obtained. On the basis of this, the two pose information can befused quickly and easily.

On the basis of the above analysis, in the embodiment of the presentapplication, after the coordinate transformation parameter isdetermined, the second pose information is transformed in coordinatebased on the coordinate transformation parameter, so as to generate thefused pose information according to the first pose information and thesecond pose information subjected to the coordinate transformation. Thefusion process is based on the same coordinate system, therebyaccelerating the fusion process, ensuring the reliability and accuracyof the fusion process, and thus ensuring the technical effect of theaccuracy and reliability of the fused pose information.

S203: generating the positioning information according to the adjacentframe images and the fused pose information.

The accuracy of the fused pose information is relatively high, and it isnot prone to be interfered by the external environment. Therefore, inthe embodiment of the present application, by generating the positioninginformation according to the fused pose information and the adjacentframe images, the problem of interference from the external environmentwhen the positioning information is generated from the adjacent frameimages can be avoided, so as to improve the accuracy and reliability ofthe positioning information.

It will be known in combination with FIG. 6 that, in some embodiments,S203 may specifically include:

S31: determining fused pose information meeting a preset error accordingto the adjacent frame images.

The adjacent frame images include image coordinate information of apreset feature point, then S31 can specifically include: inputting theimage coordinate information and the fused pose information into apreset error model, and obtaining a result outputted from the errormodel as the fused pose information meeting the preset error.

The error model includes internal parameters of the camera and externalparameters of the camera, and the external parameters of the camerainclude a rotation parameter and a displacement parameter of the camerarelative to the inertial measurement unit.

The error model is as follows:

$r_{p} = {{\frac{\rho_{j}}{\rho_{i}}R_{{BC}_{k}}R_{{WB}_{j}}R_{{WB}_{i}}R_{{BC}_{k}}P_{i}} + {\rho_{i}\left( {t_{{BC}_{k}} + {R_{{BC}_{k}}t_{{WB}_{j}}} + {R_{{BC}_{k}}R_{{WB}_{j}}t_{{WB}_{i}}} + {R_{{BC}_{k}}R_{{WB}_{j}}R_{{WB}_{i}}t_{{BC}_{k}}}} \right)} - P_{j}}$where r_(p) is the error, ρ_(j) is the inverse depth of the j-th frameimage, ρ_(i) is the inverse depth of the i-th frame image, R_(BC) _(k)is the external parameter (specifically the rotation parameter) of thecamera, R_(WB) _(j) is the rotation of the j-th frame image, R_(WB) _(i)is the rotation of the i-th frame image, P_(i) is the image coordinateinformation of the feature point in the i-th frame image, t_(BC) _(k) isthe external parameter (specifically the displacement parameter) of thecamera, t_(WB) _(j) is the displacement of the j-th image, and P_(j) isthe image coordinate information of the feature point in the j-th frameimage.

In combination with the above example, the T0 frame image is the i-thframe image, and the T1 frame image is the j-th frame image.

S32: extracting rotation information and displacement information fromthe fused pose information meeting the preset error.

That is, the rotation information is R_(WB) _(j) meeting the aboveformula, and the displacement information is t_(WB) _(j) meeting theabove formula.

In the embodiment of the present application, the rotation informationand the displacement information meeting the error are determined bycombining the relevant information of the adjacent frame images and therelevant information of the camera, so as to reduce the error about therotation information and the displacement information as much aspossible, thereby improving the accuracy and reliability of thepositioning information.

S33: determining the rotation information and the displacementinformation as the positioning information.

S204: controlling the driving of the autonomous driving vehicleaccording to the positioning information.

Regarding the description of S204, please refer to S103, and it will notbe repeated here.

In order to let readers understand the specific process of obtaining thefirst pose information and the second pose information more clearly, apositioning method according to an embodiment of the present applicationis described in detail in combination with FIG. 7 , where FIG. 7 is aflowchart diagram for a positioning method according to yet anotherembodiment of the present application.

As shown in FIG. 7 , the method includes:

S301: collecting first measurement data measured by the inertialmeasurement unit within the time period.

S302: integrating the first measurement data to generate the first poseinformation.

Based on the above examples, it can be seen that the inertialmeasurement unit can measure the posture angle, the angular velocity,the angular velocity, the rotation, the speed, the displacement andother information of the autonomous driving vehicle, and therefore thefirst measurement data is the corresponding posture angle, the angularvelocity, the angular velocity, the rotation, the speed, thedisplacement and other data of the autonomous driving vehicle collectedby the inertial measurement unit.

In an embodiment of the present application, after first measurementdata is obtained, the first measurement data is integrated to generatethe first pose information.

Generally speaking, the first measurement data is discrete data,therefore, when the first measurement data is processed by integration,part of the information can be filtered out, while the rotation, thevelocity and the displacement will be retained.

Therefore, part of the first measurement data can be filtered byintegrating the first measurement data, so as to avoid the redundancy ofsubsequent calculation, thereby improving the calculation efficiency.Moreover, since redundant information is filtered out, the accuracy andreliability of the first pose information can be ensured, therebyrealizing the reliability and accuracy of the positioning information.

Specifically, the first pose information subjected to the integrationincludes the rotation, the velocity and the displacement, which can bedenoted by the following formulas:

-   -   R_(WB) (T1)=R_(WB)(T0)Exp(ω_(BW)Δt), where R_(WB) (T1) is the        rotation corresponding to T1, R_(WB)(T0) is the rotation        corresponding to T0, and ω_(BW)Δt is the angular velocity        corresponding to (T1-T0);    -   V_(WB) (T1)=V_(WB)(T0)+α_(WB)Δt, where V_(WB)(T1) is the        velocity corresponding to T1, V_(WB)(T0) is the velocity        corresponding to T0, and α_(WB)Δt is the acceleration        corresponding to (T1−T0);    -   t_(WB)(T1)=t(T0)+V_(WB)Δt+α_(WB)Δt², where t_(WB) (T1) is the        displacement corresponding to T1, t(T0) is the displacement        corresponding to T0, V_(WB)Δt is the velocity corresponding to        (T1−T0), and α_(WB)Δt² is a square of the acceleration        corresponding to (T1-T0).

Based on the above example, if the adjacent frame images are distributedas the j-th frame image and the i-th frame image, the rotation, thespeed and the displacement corresponding to the j-th frame image can beobtained by introducing into the above formula. Specifically, therotation formula is as follows:

${R_{WB}\left( t_{j} \right)} = {{R_{WB}\left( t_{i} \right)}{\prod\limits_{k = i}^{j - 1}{{Exp}\left( {\left( {{\varpi_{BW}\left( t_{k} \right)} - {b_{g}\left( t_{k} \right)} - {\eta_{g}\left( t_{k} \right)}} \right)\Delta t} \right)}}}$

The velocity formula is as follows:

${V_{WB}\left( t_{j} \right)} = {{V_{WB}\left( t_{i} \right)} + {g_{W}\left( {t_{j} - t_{i}} \right)} + {\sum\limits_{k = i}^{j - 1}{{R_{WB}\left( t_{k} \right)}\left( {{a_{WB}\left( t_{k} \right)} - {b_{a}\left( t_{k} \right)} - {\eta_{a}\left( t_{k} \right)}} \right)\Delta t}}}$

The displacement formula is as follows:

${t_{WB}\left( t_{j} \right)} = {{t_{WB}\left( t_{i} \right)} + {\sum\limits_{k = i}^{j - 1}{{V\left( t_{k} \right)}\left( {t_{j} - t_{i}} \right)}} + {\frac{1}{2}{g_{W}\left( {t_{j} - t_{i}} \right)}^{2}} + {\frac{1}{2}{\sum\limits_{k = i}^{j - 1}{{R_{WB}\left( t_{k} \right)}\left( {{a_{WB}\left( t_{k} \right)} - {b_{a}\left( t_{k} \right)} - {\eta_{a}\left( t_{k} \right)}} \right)\Delta t^{2}}}}}$

where t_(j) is the time corresponding to the j-th frame image, t_(i) isthe time corresponding to the i-th frame image, b_(g) and b_(a) are azero offset corresponding to the inertial measurement unit, η_(g) andη_(a) are a white noise corresponding to the inertial measurement unit,Δt is a time difference between the time corresponding to the i-th frameimage and the time corresponding to the j-th frame image, and g_(W) is apreset gravity acceleration, e.g., 9.8 m/s².

S303: collecting second measurement data measured by the wheeltachometer within the time period.

S304: integrating the second measurement data to generate the secondpose information.

Similarly, in an embodiment of the present application, after the secondmeasurement data is obtained, the second measurement data is integratedto generate the second pose information.

Generally speaking, the second measurement data is discrete data,therefore, when the second measurement data is processed by integration,part of the information can be filtered out, while the velocity and thedisplacement will be retained.

Therefore, part of the second measurement data can be filtered byintegrating the second measurement data, so as to avoid the redundancyof subsequent calculation, thereby improving the calculation efficiency.Moreover, since redundant information is filtered out, the accuracy andreliability of the second pose information can be ensured, therebyrealizing the reliability and accuracy of the positioning information.

The relationship between the wheel tachometer and the inertialmeasurement unit can be denoted as follows:

-   -   V_(WB)(T1)=R_(WB)(T0) R_(BS) V_(S)−0×R_(WB)t_(BS)+ω×t_(BS) where        R_(WB) is the rotation parameter of the wheel tachometer        coordinate system relative to the inertial measurement unit        coordinate system, V_(S) is the speed measured by the wheel        tachometer, and t_(BS) is the displacement parameter of the        wheel tachometer coordinate system relative to the inertial        measurement unit coordinate system.

The second pose information subjected to integration includes thevelocity and the displacement, which can be denoted by the followingformula:

-   -   t_(WB)(T1)=∫_(T0) ^(T1) V_(WB)dτ, where t_(WB)(T1) is the        displacement corresponding to T1, and V_(WB) is the velocity.

S305: generating the positioning information according to the first poseinformation, the second pose information and the adjacent frame images.

Regarding the description of S305, please refer to S102, or refer toS202 to S203, which will not be repeated here.

S306: controlling the driving of the autonomous driving vehicleaccording to positioning information.

S307: regarding the description of S307, please refer to S103, whichwill not be repeated here.

According to another aspect of embodiments of the present application,the embodiment of the present application further provides a positioningapparatus corresponding to the above method embodiments, so as torealize the positioning method described in any of the aboveembodiments.

Referring to FIG. 8 , FIG. 8 is a schematic diagram of a positioningapparatus according to an embodiment of the present application.

As shown in FIG. 8 , the apparatus includes:

-   -   a collecting module 11, configured to collect first pose        information measured by an inertial measurement unit within a        preset time period, and collect second pose information measured        by a wheel tachometer within the time period, where the time        period is a sampling time interval when a camera collects        adjacent frame images;    -   a generating module 12, configured to generate positioning        information according to the first pose information, the second        pose information and the adjacent frame images;    -   a controlling module 13, configured to control driving of an        autonomous driving vehicle according to the positioning        information.

In some embodiments, the generating module 12 is configured to generatefused pose information by fusing the first pose information and thesecond pose information, and to generate the positioning informationaccording to the adjacent frame images and the fused pose information.

In some embodiments, the generating module 12 is configured to obtain acoordinate transformation parameter of the wheel tachometer relative tothe inertial measurement unit, to perform a coordinate transformation onthe second pose information according to the coordinate transformationparameter, and to fuse the first pose information and the second poseinformation subjected to the coordinate transformation to generate thefused pose information.

In some embodiments, the generating module 12 is configured to determinefused pose information meeting a preset error according to the adjacentframe images, to extract rotation information and displacementinformation from the fused pose information meeting the preset error,and to determine the rotation information and the displacementinformation as the positioning information.

In some embodiments, the adjacent frame images include image coordinateinformation of a preset feature point; and the generating module 12 isconfigured to input the image coordinate information and the fused poseinformation into a preset error model, and obtain a result outputtedfrom the error mode as the fused pose information meeting the preseterror.

In some embodiments, the error model includes an internal parameter ofthe camera and an external parameter of the camera, and the externalparameter of the camera includes a rotation parameter and a displacementparameter of the camera relative to the inertial measurement unit.

In some embodiments, the collecting module 11 is configured to collectfirst measurement data measured by the inertial measurement unit withinthe time period, and to integrate the first measurement data to generatethe first pose information.

In some embodiments, the collecting module 11 is configured to collectsecond measurement data measured by the wheel tachometer within the timeperiod, and to integrate the second measurement data to generate thesecond pose information.

According to embodiments of the present application, the presentapplication further provides an electronic device and a readable storagemedium.

Referring to FIG. 9 , FIG. 9 is a block diagram of an electronic deviceaccording to an embodiment of the present application.

The electronic device is intended to represent various forms of digitalcomputers, such as a laptop computer, a desktop computer, a workstation,a personal digital assistant, a server, a blade server, a mainframecomputer, and other suitable computers. The electronic device may alsorepresent various forms of mobile devices, such as a personal digitalassistant, a cellular phone, a smart phone, a wearable device, and othersimilar computing devices. Components shown herein, connections andrelationships thereof, as well as functions thereof are merely examplesand are not intended to limit the present application implementationdescribed and/or claimed herein.

As shown in FIG. 9 , the electronic device includes: one or moreprocessors 101, memory 102, and interfaces for connecting variouscomponents, including a high-speed interface and a low-speed interface.The various components are interconnected through different buses andcan be installed on a common motherboard or be installed in other waysas required. The processor may process instructions executed within theelectronic device, where the instructions include instructions stored inor on a memory to display graphical information of the GUI on anexternal input/output device (such as, a display device coupled to aninterface). In other embodiments, a plurality of processors and/or aplurality of buses may be used with a plurality of memories, ifrequired. Similarly, a plurality of electronic devices can be connected,each of which provides some of the necessary operations (for example,functions as a server array, a set of blade servers, or a multiprocessorsystem). In FIG. 9 , one processor 101 is taken as an example.

The memory 102 is a non-transitory computer-readable storage mediumprovided in the present application. The memory stores instructionsexecutable by at least one processor to cause the at least one processorto perform the positioning method provided by the embodiment of thepresent application. The non-transitory computer-readable storage mediumof the present application stores computer instructions, where thecomputer instructions are configured to cause a computer to perform thepositioning method provided by the embodiment of the presentapplication.

The memory 102, as a non-transitory computer-readable storage medium,can be configured to store a non-transitory software program, anon-transitory computer executable program and module, such as a programinstruction/module in the embodiments of the present application. Byrunning the non-transitory software program, instructions and modulesstored in the memory 102, the processor 601 performs various functionalapplications and data processing of the server, that is, realizes thepositioning method in the above method embodiments.

The memory 102 may include a program storing area and a data storingarea, where the program storing area may store an operating system andapplication programs required by at least one function; and the datastoring area may store data created according to the use of theelectronic device and the like. In addition, the memory 102 may includea high-speed random access memory, and may also include a non-transitorymemory, such as at least one disk storage device, a flash memory device,or other non-transitory solid-state memory devices. In some embodiments,the memory 102 may optionally include memories provided remotely withrespect to the processor 101, and these remote memories may be connectedvia a network to an electronic device. Examples of the above-mentionednetwork may include, but are not limited to, Internet, an intranet, alocal area network, a block-chain-based service network (BSN), a mobilecommunication network and a combination thereof.

The electronic device may further include: an input device 103 and anoutput device 104. The processor 101, the memory 102, the input device103 and the output device 104 may be connected via a bus or other means,and an example of a connection via the bus is shown in FIG. 9 .

The input device 103 may receive inputted digital or personalinformation, and generate key signal input related to a user setting andfunctional control of the electronic device. The input device, forexample, is a touch screen, a keypad, a mouse, a trackpad, a touchpad, apointer, one or more mouse buttons, a trackball, a joystick and otherinput devices. The output device 104 may include: a display device, anauxiliary lighting device (e.g., an LED), a tactile feedback device(e.g., a vibration motor) and the like. The display device may include,but is not limited to, a liquid crystal display (LCD), a light emittingdiode (LED) display and a plasma display. In some embodiments, thedisplay device may be a touch screen.

Various embodiments of the systems and technologies described herein maybe implemented in a digital electronic circuit system, an integratedcircuit system, a specialized ASIC (application specific integratedcircuits), computer hardware, firmware, software, and/or a combinationthereof. These various embodiments may include: being implemented in oneor more computer programs, where the one or more computer programs maybe executed and/or interpreted on a programmable system including atleast one programmable processor, where the programmable processor maybe a specialized or general-purpose programmable processor, which mayreceive data and instructions from a storage system, at least one inputdevice and at least one output device and send the data and instructionsto the storage system, the at least one input device and the at leastone output device.

These computer programs (also referred to as programs, software,software applications, or code) include machine instructions forprogrammable processors and can be implemented by using a high-levelprocedure and/or object-oriented programming language, and/or anassembly/machine language. As used herein, the terms “machine-readablemedium” and “computer-readable medium” refer to any computer programproduct, apparatus, and/or device (e.g., a magnetic disk, an opticaldisk, a memory, a programmable logic device (PLD)) for providing machineinstructions and/or data to the programmable processor, and include amachine-readable medium that receives machine instructions asmachine-readable signals. The term “machine-readable signal” refers toany signal configured to provide machine instructions and/or data to theprogrammable processor.

In order to provide interaction with an user, the systems and techniquesdescribed herein may be implemented on a computer, where the computerhas: a display device (e.g., a CRT (cathode ray tube) or a LCD (liquidcrystal display) monitor) for displaying information to the user; and akeyboard and a pointing device (e.g., a mouse or a trackball), throughwhich the user can provide input to a computer. Other types of devicesmay also be used to provide interaction with the user; for example, thefeedback provided to the user may be any form of sensing feedback (suchas, visual feedback, auditory feedback, or tactile feedback); and theinput from the user may be received in any form (including acousticinput, voice input, or tactile input).

The systems and technologies described here may be implemented in acomputing system (e.g., a data server) including a back-end component,or in a computing system (e.g., an application server) including amiddleware component, or in a computing system (e.g., a user computerhaving a graphical user interface or a web browser, through which theuser can interact with the implementation of the systems andtechnologies described herein) including a front-end component, or in acomputing system including any combination of the background component,the middleware component, or the front-end component. The components ofthe system may be interconnected via digital data communication (e.g., acommunication network) in any form or medium. Examples of thecommunication network include: a local area network (LAN), ablock-chain-based service network (BSN), a wide area network (WAN) andInternet.

The computer system may include a client and a server. The client andthe server are generally located far away from each other and usuallyinteract with each other through a communication network. A relationshipbetween the client and the server is generated by computer programsrunning on corresponding computers and having a client-serverrelationship between each other.

According to another aspect of embodiments of the present application,the embodiments of the present application further provides anautonomous driving vehicle, where the autonomous driving vehicleincludes the positioning apparatus described in the above embodiments,or includes the electronic device described in the above embodiments.

According to another aspect of embodiments of the present application,the embodiments of the present application further provide a positioningmethod.

Referring to FIG. 10 , FIG. 10 is a flowchart diagram of a positioningmethod of yet another embodiment of the present application.

As shown in FIG. 10 , the method includes:

S1: collecting respective pose information measured by at least twosensors within a preset time period, where the time period is a samplingtime interval when a camera collects adjacent frame images.

The number of sensors is multiple, and one sensor corresponds to onekind of pose information. In other words, each sensor collects the poseinformation of the autonomous driving vehicle.

The sensor can also be an inertial measurement unit, a wheel tachometer,a radar sensor and the like, which will not be listed here.

Specifically, when the number of sensors is two, one sensor can be theinertial measurement unit described in the above examples, and therespective pose information is the first pose information in the aboveexamples; and the other sensor can be the wheel tachometer described inthe above examples, and the respective pose information is the secondpose information in the above examples.

S2: generating positioning information according to the respective poseinformation and the adjacent frame images.

In the embodiments of the present application, it is equivalent torectifying, according to the respective pose information, thepositioning information corresponding to the adjacent frame images, soas to generate the positioning information with higher reliability. Forspecific process, please refer to the above examples, which will not berepeated here.

It should be understood that steps can be reordered, added, or deletedusing the various forms of processes shown above. For example, the stepsrecited in the present application can be performed in parallel, insequence or in different orders, as long as expected results of thetechnical solution disclosed by the present application can be realized,and there is no limitation herein.

The above specific implementations do not limit the protection scope ofthe present application. It should be understood by those skilled in theart that various modifications, combinations, sub-combinations andsubstitutions may be made according to design requirements and otherfactors. Any modification, equivalent replacement and improvement madewithin the spirit and principle of the present application shall beincluded in the protection scope of the present application.

What is claimed is:
 1. A positioning method, wherein the method isapplied to an autonomous driving vehicle, and the method comprises:collecting first pose information measured by an inertial measurementunit within a preset time period, and collecting second pose informationmeasured by a wheel tachometer within the time period, wherein the timeperiod is a sampling time interval when a camera collects adjacent frameimages; generating positioning information according to the first poseinformation, the second pose information and the adjacent frame images;and controlling driving of the autonomous driving vehicle according tothe positioning information; wherein the generating the positioninginformation according to the first pose information, the second poseinformation and the adjacent frame images comprises: generating fusedpose information by fusing the first pose information and the secondpose information; and generating the positioning information accordingto the adjacent frame images and the fused pose information; wherein thegenerating the positioning information according to the adjacent frameimages and the fused pose information comprises: determining fused poseinformation that is in line with a preset error according to theadjacent frame images; and extracting rotation information anddisplacement information from the fused pose information that is in linewith the preset error, and determining the rotation information and thedisplacement information as the positioning information; wherein theadjacent frame images comprise image coordinate information of a presetfeature point; and the determining fused pose information that is inline with a preset error according to the adjacent frame imagescomprises: inputting the image coordinate information and the fused poseinformation into a preset error model; and obtaining a result outputtedfrom the error model as the fused pose information that is in line withthe preset error; wherein the error model comprises an internalparameter of the camera and an external parameter of the camera, and theexternal parameter of the camera comprises a rotation parameter and adisplacement parameter of the camera relative to the inertialmeasurement unit.
 2. The method according to claim 1, wherein thegenerating the fused pose information by fusing the first poseinformation and the second pose information comprises: obtaining acoordinate transformation parameter of the wheel tachometer relative tothe inertial measurement unit; performing a coordinate transformation onthe second pose information according to the coordinate transformationparameter; and fusing the first pose information and the second poseinformation subjected to the coordinate transformation to generate thefused pose information.
 3. The method according to claim 1, wherein thecollecting the first pose information measured by the inertialmeasurement unit within the preset time period comprises: collectingfirst measurement data measured by the inertial measurement unit withinthe time period; and integrating the first measurement data to generatethe first pose information.
 4. The method according to claim 1, whereinthe collecting the second pose information measured by the wheeltachometer within the time period comprises: collecting secondmeasurement data measured by the wheel tachometer within the timeperiod; and integrating the second measurement data to generate thesecond pose information.
 5. A positioning apparatus, wherein theapparatus comprises: at least one processor; and a memory connected withthe at least one processor in communication, wherein, the memory storesinstructions executable by the at least one processor, wherein theinstructions are executed by the at least one processor to cause the atleast one processor to: collect first pose information measured by aninertial measurement unit within a preset time period, and collectsecond pose information measured by a wheel tachometer within the timeperiod, wherein the time period is a sampling time interval when acamera collects adjacent frame images; generate positioning informationaccording to the first pose information, the second pose information andthe adjacent frame images; and control driving of the autonomous drivingvehicle according to the positioning information; wherein the at leastone processor is further enabled to: generate fused pose information byfusing the first pose information and the second pose information; andgenerate the positioning information according to the adjacent frameimages and the fused pose information; wherein the at least oneprocessor is further enabled to: determine fused pose information thatis in line with a preset error according to the adjacent frame images;extract rotation information and displacement information from the fusedpose information that is in line with the preset error, and determinethe rotation information and the displacement information as thepositioning information; wherein the adjacent frame images compriseimage coordinate information of a preset feature point; the at least oneprocessor is further enabled to input the image coordinate informationand the fused pose information into a preset error model; and obtain aresult outputted from the error model as the fused pose information thatis in line with the preset error; wherein the error model comprises aninternal parameter of the camera and an external parameter of thecamera, and the external parameter of the camera comprises a rotationparameter and a displacement parameter of the camera relative to theinertial measurement unit.
 6. The apparatus according to claim 5,wherein the at least one processor is further enabled to: obtain acoordinate transformation parameter of the wheel tachometer relative tothe inertial measurement unit; perform a coordinate transformation onthe second pose information according to the coordinate transformationparameter; and fuse the first pose information and the second poseinformation subjected to the coordinate transformation to generate thefused pose information.
 7. The apparatus according to claim 5, whereinthe at least one processor is further enabled to collect firstmeasurement data measured by the inertial measurement unit within thetime period; and integrate the first measurement data to generate thefirst pose information.
 8. The apparatus according to claim 5, whereinthe at least one processor is further enabled to collect secondmeasurement data measured by the wheel tachometer within the timeperiod, and integrate the second measurement data to generate the secondpose information.
 9. A non-transitory computer-readable storage mediumfor storing computer instructions, wherein the computer instructions areconfigured to cause a computer to perform the following steps:collecting first pose information measured by an inertial measurementunit within a preset time period, and collecting second pose informationmeasured by a wheel tachometer within the time period, wherein the timeperiod is a sampling time interval when a camera collects adjacent frameimages; generating positioning information according to the first poseinformation, the second pose information and the adjacent frame images;and controlling driving of the autonomous driving vehicle according tothe positioning information; wherein the computer instructions arefurther configured to cause the computer to perform the following steps:generating fused pose information by fusing the first pose informationand the second pose information; and generating the positioninginformation according to the adjacent frame images and the fused poseinformation; wherein the computer instructions are further configured tocause the computer to perform the following steps: determining fusedpose information that is in line with a preset error according to theadjacent frame images; and extracting rotation information anddisplacement information from the fused pose information that is in linewith the preset error, and determining the rotation information and thedisplacement information as the positioning information; wherein theadjacent frame images comprise image coordinate information of a presetfeature point; and the computer instructions are further configured tocause the computer to perform the following steps: inputting the imagecoordinate information and the fused pose information into a preseterror model; and obtaining a result outputted from the error model asthe fused pose information that is in line with the preset error;wherein the error model comprises an internal parameter of the cameraand an external parameter of the camera, and the external parameter ofthe camera comprises a rotation parameter and a displacement parameterof the camera relative to the inertial measurement unit.
 10. Thenon-transitory computer-readable storage medium according to claim 9,wherein the computer instructions are further configured to cause thecomputer to perform the following steps: obtaining a coordinatetransformation parameter of the wheel tachometer relative to theinertial measurement unit; performing a coordinate transformation on thesecond pose information according to the coordinate transformationparameter; and fusing the first pose information and the second poseinformation subjected to the coordinate transformation to generate thefused pose information.